{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 一、天气以及时间分类\n",
    "比赛地址： [https://www.datafountain.cn/competitions/555](https://www.datafountain.cn/competitions/555)\n",
    "\n",
    "![](https://wid.s3.cn-north-1.amazonaws.com.cn/uploads/images/2021-11-18/%E5%A4%A9%E6%B0%94%E4%BB%A5%E5%8F%8A%E6%97%B6%E9%97%B4%E5%88%86%E7%B1%BB-711628.png)\n",
    "\n",
    "### 1.赛题背景\n",
    "在自动驾驶场景中，天气和时间（黎明、早上、下午、黄昏、夜晚）会对传感器的精度造成影响，比如雨天和夜晚会对视觉传感器的精度造成很大的影响。此赛题旨在对拍摄的照片天气和时间进行分类，从而在不同的天气和时间使用不同的自动驾驶策略。\n",
    "\n",
    "### 2.赛题任务\n",
    "此赛题的数据集由云测数据提供。比赛数据集中包含3000张真实场景下行车记录仪采集的图片，其中训练集包含2600张带有天气和时间类别标签的图片，测试集包含400张不带有标签的图片。参赛者需基于Oneflow框架在训练集上进行训练，对测试集中照片的天气和时间进行分类。\n",
    "\n",
    "### 3.数据简介\n",
    "本赛题的数据集包含2600张人工标注的天气和时间标签。\n",
    "* 天气类别：多云、晴天、雨天、雪天和雾天5个类别\n",
    "* 时间：黎明、早上、下午、黄昏、夜晚5个类别\n",
    "\n",
    "![](https://ai-studio-static-online.cdn.bcebos.com/03ed0a6b02cd4de4b5236db247b2e1226499f3a12bd74a4eb67f54c187483370)\n",
    "\n",
    "下午 多云\n",
    "\n",
    "![](https://ai-studio-static-online.cdn.bcebos.com/02972ce038f3482297b032188e81279b4cfc5c32bd464b518b81aa1d9f7a1524)\n",
    "\n",
    "早上 雨天\n",
    "\n",
    "### 4.数据说明\n",
    "数据集包含anno和image两个文件夹，anno文件夹中包含2600个标签json文件，image文件夹中包含3000张行车记录仪拍摄的JPEG编码照片。图片标签将字典以json格式序列化进行保存：\n",
    "\n",
    "<table>\n",
    "<thead>\n",
    "<tr>\n",
    "<th>列名</th>\n",
    "<th>取值范围</th>\n",
    "<th>作用</th>\n",
    "</tr>\n",
    "</thead>\n",
    "<tbody>\n",
    "<tr>\n",
    "<td>Period</td>\n",
    "<td>黎明、早上、下午、黄昏、夜晚</td>\n",
    "<td>图片拍摄时间</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>Weather</td>\n",
    "<td>多云、晴天、雨天、雪天、雾天</td>\n",
    "<td>图片天气</td>\n",
    "</tr>\n",
    "</tbody>\n",
    "</table>\n",
    "\n",
    "### 5.提交要求\n",
    "参赛者使用Oneflow框架对数据集进行训练后对测试集图片进行推理后，\n",
    "1.将测试集图片的目标检测和识别结果以与训练集格式保持一致的json文件序列化保存，并上传至参赛平台由参赛平台自动评测返回结果。\n",
    "2.在提交时的备注附上自己的模型github仓库链接\n",
    "\n",
    "### 6.提交示例\n",
    "{\n",
    "    “annotations”: [\n",
    "        {\n",
    "            “filename”: “test_images\\00008.jpg”,\n",
    "            “period”: “Morning”,\n",
    "            “weather”: “Cloudy”\n",
    "        }]\n",
    "}\n",
    "\n",
    "\n",
    "### 7.解题思路\n",
    "总体上看，该任务可以分为2个：一个是预测时间、一个是预测天气，具体如下：\n",
    "\n",
    "* 预测时间、天气数据标签列表生成\n",
    "* 数据集划分\n",
    "* 数据均衡（数据很不均衡）\n",
    "* 分别预测\n",
    "* 合并预测结果"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 二、数据集准备\n",
    "\n",
    "### 1.数据下载"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2022-01-17 11:05:49--  https://awscdn.datafountain.cn/cometition_data2/Files/BDCI2021/555/train_dataset.zip\n",
      "Resolving awscdn.datafountain.cn (awscdn.datafountain.cn)... 210.51.40.148, 210.51.40.156, 210.51.40.133, ...\n",
      "Connecting to awscdn.datafountain.cn (awscdn.datafountain.cn)|210.51.40.148|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 419324853 (400M) [application/octet-stream]\n",
      "Saving to: ‘train_dataset.zip’\n",
      "\n",
      "train_dataset.zip   100%[===================>] 399.90M   102MB/s    in 4.3s    \n",
      "\n",
      "2022-01-17 11:05:54 (94.0 MB/s) - ‘train_dataset.zip’ saved [419324853/419324853]\n",
      "\n",
      "--2022-01-17 11:05:54--  https://awscdn.datafountain.cn/cometition_data2/Files/BDCI2021/555/test_dataset.zip\n",
      "Resolving awscdn.datafountain.cn (awscdn.datafountain.cn)... 210.51.40.156, 210.51.40.150, 210.51.40.133, ...\n",
      "Connecting to awscdn.datafountain.cn (awscdn.datafountain.cn)|210.51.40.156|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 62269247 (59M) [application/octet-stream]\n",
      "Saving to: ‘test_dataset.zip’\n",
      "\n",
      "test_dataset.zip    100%[===================>]  59.38M  65.4MB/s    in 0.9s    \n",
      "\n",
      "2022-01-17 11:05:55 (65.4 MB/s) - ‘test_dataset.zip’ saved [62269247/62269247]\n",
      "\n",
      "--2022-01-17 11:05:55--  https://awscdn.datafountain.cn/cometition_data2/Files/BDCI2021/555/submit_example.json\n",
      "Resolving awscdn.datafountain.cn (awscdn.datafountain.cn)... 210.51.40.156, 210.51.40.148, 210.51.40.150, ...\n",
      "Connecting to awscdn.datafountain.cn (awscdn.datafountain.cn)|210.51.40.156|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 593 [application/octet-stream]\n",
      "Saving to: ‘submit_example.json’\n",
      "\n",
      "submit_example.json 100%[===================>]     593  --.-KB/s    in 0s      \n",
      "\n",
      "2022-01-17 11:05:56 (187 MB/s) - ‘submit_example.json’ saved [593/593]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# 直接下载，速度超快\n",
    "!wget https://awscdn.datafountain.cn/cometition_data2/Files/BDCI2021/555/train_dataset.zip\n",
    "!wget https://awscdn.datafountain.cn/cometition_data2/Files/BDCI2021/555/test_dataset.zip\n",
    "!wget https://awscdn.datafountain.cn/cometition_data2/Files/BDCI2021/555/submit_example.json"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 2.数据解压缩"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 解压缩数据集\n",
    "!unzip -qoa test_dataset.zip \n",
    "!unzip -qoa train_dataset.zip"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 3.按时间制作标签\n",
    "注意事项：虽然数据描述说时间** Period 为\t黎明、早上、下午、黄昏、夜晚**，但是经过遍历发现只有4类。。。。。，故如下制作标签"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/aistudio\n",
      "写入train_period.txt完成！！！\n"
     ]
    }
   ],
   "source": [
    "# 标签修改\n",
    "%cd ~\n",
    "import json\n",
    "import os\n",
    "\n",
    "train = {}\n",
    "with open('train.json', 'r') as f:\n",
    "    train = json.load(f)\n",
    "\n",
    "period_list = {'Dawn': 0, 'Dusk': 1, 'Morning': 2, 'Afternoon': 3}\n",
    "f_period=open('train_period.txt','w')\n",
    "for item in train[\"annotations\"]:\n",
    "    label = period_list[item['period']] \n",
    "    file_name=os.path.join(item['filename'].split('\\\\')[0], item['filename'].split('\\\\')[1])\n",
    "    f_period.write(file_name +' '+ str(label) +'\\n')\n",
    "f_period.close()\n",
    "print(\"写入train_period.txt完成！！！\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 4.数据集划分并数据均衡"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/aistudio\n",
      "2    1613\n",
      "3     829\n",
      "1     124\n",
      "0      34\n",
      "Name: 1, dtype: int64\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7feffe438b50>"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD4CAYAAADlwTGnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAEbZJREFUeJzt3X+s3fV93/HnK/ZgTaKFH75lxHZqa3ETkapdyS1hijbRegOTVDV/pBmoC25GZ22DNi3VEqfVhNoqGt2msaJmaF5xA1IEQywtVuONuiRZNK0QLmkKAZJyR0h8LYhvCiHbWEMd3v3jfFhOLr6+vucc34P9eT6kq/v9vj+f7/m+zxH4db/f7znnm6pCktSf10y7AUnSdBgAktQpA0CSOmUASFKnDABJ6pQBIEmdMgAkqVMGgCR1ygCQpE6tn3YDx7Nhw4basmXLtNuQpFPKQw899I2qmllp3qs6ALZs2cLc3Ny025CkU0qSr57IPE8BSVKnDABJ6pQBIEmdMgAkqVMGgCR1ygCQpE4ZAJLUqRUDIMm+JEeSfHFJ/eeTfCnJo0n+9VD9w0nmk3w5yWVD9R2tNp9kz2SfhiRptU7kg2AfA34buP3lQpIfB3YCP1JV307y/a1+AXAl8DbgjcAfJfnBttlHgX8ALAAPJtlfVY9N6omcqC17PrnWuxzJUze+e9otSDrNrRgAVfXZJFuWlP8ZcGNVfbvNOdLqO4E7W/0rSeaBi9rYfFU9CZDkzjZ3zQNAkjQw6jWAHwT+bpIHkvz3JD/W6huBQ0PzFlptuforJNmdZC7J3OLi4ojtSZJWMmoArAfOAS4G/gVwV5JMoqGq2ltVs1U1OzOz4ncZSZJGNOqXwS0An6iqAj6X5CVgA3AY2Dw0b1OrcZy6JGkKRj0C+H3gxwHaRd4zgG8A+4Erk5yZZCuwDfgc8CCwLcnWJGcwuFC8f9zmJUmjW/EIIMkdwCXAhiQLwA3APmBfe2voi8CudjTwaJK7GFzcPQpcW1XfaY9zHXAvsA7YV1WPnoTnI0k6QSfyLqCrlhn6R8vM/wjwkWPUDwAHVtWdJOmk8ZPAktQpA0CSOmUASFKnDABJ6pQBIEmdMgAkqVMGgCR1ygCQpE4ZAJLUKQNAkjplAEhSpwwASeqUASBJnTIAJKlTBoAkdcoAkKROrRgASfYlOdLu/rV07JeTVJINbT1Jbk4yn+ThJBcOzd2V5In2s2uyT0OStFoncgTwMWDH0mKSzcClwNeGypczuA/wNmA3cEubew6DW0m+A7gIuCHJ2eM0Lkkaz4oBUFWfBZ49xtBNwAeBGqrtBG6vgfuBs5KcD1wGHKyqZ6vqOeAgxwgVSdLaGekaQJKdwOGq+tMlQxuBQ0PrC622XF2SNCUr3hR+qSSvBX6FwemfiUuym8HpI970pjedjF1IkhjtCOBvAVuBP03yFLAJ+HySvwkcBjYPzd3UasvVX6Gq9lbVbFXNzszMjNCeJOlErDoAquqRqvr+qtpSVVsYnM65sKqeAfYDV7d3A10MPF9VTwP3ApcmObtd/L201SRJU3IibwO9A/hj4C1JFpJcc5zpB4AngXngPwH/HKCqngV+A3iw/fx6q0mSpmTFawBVddUK41uGlgu4dpl5+4B9q+xPknSS+ElgSeqUASBJnTIAJKlTBoAkdcoAkKROGQCS1CkDQJI6ZQBIUqcMAEnqlAEgSZ0yACSpUwaAJHXKAJCkThkAktQpA0CSOmUASFKnDABJ6tSJ3BJyX5IjSb44VPs3Sb6U5OEkv5fkrKGxDyeZT/LlJJcN1Xe02nySPZN/KpKk1TiRI4CPATuW1A4CP1RVPwz8GfBhgCQXAFcCb2vb/Ick65KsAz4KXA5cAFzV5kqSpmTFAKiqzwLPLqn9YVUdbav3A5va8k7gzqr6dlV9hcHN4S9qP/NV9WRVvQjc2eZKkqZkEtcA/jHwX9vyRuDQ0NhCqy1Xf4Uku5PMJZlbXFycQHuSpGMZKwCS/CpwFPj4ZNqBqtpbVbNVNTszMzOph5UkLbF+1A2T/Czwk8D2qqpWPgxsHpq2qdU4Tl2SNAUjHQEk2QF8EPipqnphaGg/cGWSM5NsBbYBnwMeBLYl2ZrkDAYXiveP17okaRwrHgEkuQO4BNiQZAG4gcG7fs4EDiYBuL+q/mlVPZrkLuAxBqeGrq2q77THuQ64F1gH7KuqR0/C85EknaAVA6CqrjpG+dbjzP8I8JFj1A8AB1bVnSTppPGTwJLUKQNAkjplAEhSpwwASeqUASBJnTIAJKlTBoAkdcoAkKROGQCS1CkDQJI6ZQBIUqcMAEnqlAEgSZ0yACSpUwaAJHXKAJCkThkAktSpFQMgyb4kR5J8cah2TpKDSZ5ov89u9SS5Ocl8koeTXDi0za42/4kku07O05EknagTOQL4GLBjSW0PcF9VbQPua+sAlzO4Efw2YDdwCwwCg8G9hN8BXATc8HJoSJKmY8UAqKrPAs8uKe8EbmvLtwFXDNVvr4H7gbOSnA9cBhysqmer6jngIK8MFUnSGhr1GsB5VfV0W34GOK8tbwQODc1baLXl6q+QZHeSuSRzi4uLI7YnSVrJ2BeBq6qAmkAvLz/e3qqararZmZmZST2sJGmJUQPg6+3UDu33kVY/DGwemrep1ZarS5KmZNQA2A+8/E6eXcA9Q/Wr27uBLgaeb6eK7gUuTXJ2u/h7aatJkqZk/UoTktwBXAJsSLLA4N08NwJ3JbkG+Crw3jb9APAuYB54AXg/QFU9m+Q3gAfbvF+vqqUXliVJa2jFAKiqq5YZ2n6MuQVcu8zj7AP2rao7SdJJ4yeBJalTBoAkdcoAkKROGQCS1CkDQJI6ZQBIUqcMAEnqlAEgSZ0yACSpUwaAJHXKAJCkThkAktQpA0CSOmUASFKnDABJ6pQBIEmdWvGGMMeT5JeAn2NwU/hHGNwB7HzgTuBc4CHgfVX1YpIzgduBtwN/DvzDqnpqnP1r+rbs+eS0WzghT9347mm3IL3qjHwEkGQj8AvAbFX9ELAOuBL4TeCmqnoz8BxwTdvkGuC5Vr+pzZMkTcm4p4DWA9+XZD3wWuBp4CeAu9v4bcAVbXlnW6eNb0+SMfcvSRrRyAFQVYeBfwt8jcE//M8zOOXzzao62qYtABvb8kbgUNv2aJt/7qj7lySNZ5xTQGcz+Kt+K/BG4HXAjnEbSrI7yVySucXFxXEfTpK0jHFOAf194CtVtVhVfwl8AngncFY7JQSwCTjclg8DmwHa+BsYXAz+HlW1t6pmq2p2ZmZmjPYkScczTgB8Dbg4yWvbufztwGPAp4H3tDm7gHva8v62Thv/VFXVGPuXJI1hnGsADzC4mPt5Bm8BfQ2wF/gQcH2SeQbn+G9tm9wKnNvq1wN7xuhbkjSmsT4HUFU3ADcsKT8JXHSMuX8B/PQ4+5MkTY6fBJakThkAktQpA0CSOmUASFKnDABJ6pQBIEmdMgAkqVMGgCR1ygCQpE4ZAJLUKQNAkjplAEhSpwwASeqUASBJnTIAJKlTBoAkdcoAkKROjRUASc5KcneSLyV5PMnfSXJOkoNJnmi/z25zk+TmJPNJHk5y4WSegiRpFOMeAfwW8N+q6q3AjwCPM7jX731VtQ24j+/e+/dyYFv72Q3cMua+JUljGDkAkrwB+Hu0m75X1YtV9U1gJ3Bbm3YbcEVb3gncXgP3A2clOX/kziVJYxnnCGArsAj8bpI/SfI7SV4HnFdVT7c5zwDnteWNwKGh7Rda7Xsk2Z1kLsnc4uLiGO1Jko5nnABYD1wI3FJVPwr8X757ugeAqiqgVvOgVbW3qmaranZmZmaM9iRJxzNOACwAC1X1QFu/m0EgfP3lUzvt95E2fhjYPLT9plaTJE3ByAFQVc8Ah5K8pZW2A48B+4FdrbYLuKct7weubu8Guhh4fuhUkSRpja0fc/ufBz6e5AzgSeD9DELlriTXAF8F3tvmHgDeBcwDL7S5kqQpGSsAquoLwOwxhrYfY24B146zP0nS5PhJYEnqlAEgSZ0yACSpUwaAJHXKAJCkThkAktQpA0CSOmUASFKnDABJ6pQBIEmdMgAkqVMGgCR1ygCQpE4ZAJLUKQNAkjplAEhSpwwASerU2AGQZF2SP0nyB219a5IHkswn+c/tdpEkObOtz7fxLePuW5I0ukkcAXwAeHxo/TeBm6rqzcBzwDWtfg3wXKvf1OZJkqZkrABIsgl4N/A7bT3ATwB3tym3AVe05Z1tnTa+vc2XJE3BuEcA/x74IPBSWz8X+GZVHW3rC8DGtrwROATQxp9v879Hkt1J5pLMLS4ujtmeJGk5IwdAkp8EjlTVQxPsh6raW1WzVTU7MzMzyYeWJA1ZP8a27wR+Ksm7gL8O/A3gt4Czkqxvf+VvAg63+YeBzcBCkvXAG4A/H2P/kqQxjHwEUFUfrqpNVbUFuBL4VFX9DPBp4D1t2i7gnra8v63Txj9VVTXq/iVJ4zkZnwP4EHB9knkG5/hvbfVbgXNb/Xpgz0nYtyTpBI1zCuj/q6rPAJ9py08CFx1jzl8APz2J/UmSxucngSWpUwaAJHXKAJCkThkAktQpA0CSOmUASFKnDABJ6pQBIEmdMgAkqVMGgCR1ygCQpE4ZAJLUKQNAkjplAEhSpwwASeqUASBJnRrnpvCbk3w6yWNJHk3ygVY/J8nBJE+032e3epLcnGQ+ycNJLpzUk5Akrd44RwBHgV+uqguAi4Frk1zA4FaP91XVNuA+vnvrx8uBbe1nN3DLGPuWJI1pnJvCP11Vn2/L/xt4HNgI7ARua9NuA65oyzuB22vgfuCsJOeP3LkkaSwTuQaQZAvwo8ADwHlV9XQbegY4ry1vBA4NbbbQapKkKRg7AJK8HvgvwC9W1beGx6qqgFrl4+1OMpdkbnFxcdz2JEnLGCsAkvw1Bv/4f7yqPtHKX3/51E77faTVDwObhzbf1Grfo6r2VtVsVc3OzMyM054k6TjGeRdQgFuBx6vq3w0N7Qd2teVdwD1D9avbu4EuBp4fOlUkSVpj68fY9p3A+4BHknyh1X4FuBG4K8k1wFeB97axA8C7gHngBeD9Y+xbkjSmkQOgqv4HkGWGtx9jfgHXjro/SdJk+UlgSeqUASBJnTIAJKlTBoAkdcoAkKROjfM2UEkTtmXPJ6fdwgl56sZ3T7sFTYBHAJLUKQNAkjplAEhSpwwASeqUASBJnTIAJKlTBoAkdcoAkKROGQCS1CkDQJI65VdBSDot+bUaK1vzI4AkO5J8Ocl8kj1rvX9J0sCaBkCSdcBHgcuBC4Crklywlj1IkgbW+gjgImC+qp6sqheBO4Gda9yDJAnI4F7ta7Sz5D3Ajqr6ubb+PuAdVXXd0JzdwO62+hbgy2vW4Og2AN+YdhOnEV/PyfL1nJxT5bX8gaqaWWnSq+4icFXtBfZOu4/VSDJXVbPT7uN04es5Wb6ek3O6vZZrfQroMLB5aH1Tq0mS1thaB8CDwLYkW5OcAVwJ7F/jHiRJrPEpoKo6muQ64F5gHbCvqh5dyx5OklPqlNUpwNdzsnw9J+e0ei3X9CKwJOnVw6+CkKROGQCS1CkDQJI69ar7HID6k+QioKrqwfbVIDuAL1XVgSm3ps4leSuDbyvY2EqHgf1V9fj0upocjwBGkOStSbYnef2S+o5p9XSqSnIDcDNwS5J/Bfw28DpgT5JfnWpz6lqSDzH4upoAn2s/Ae44Xb7I0ncBrVKSXwCuBR4H/jbwgaq6p419vqounGZ/p5okjzB4Hc8EngE2VdW3knwf8EBV/fBUGzyNJHl/Vf3utPs4VST5M+BtVfWXS+pnAI9W1bbpdDY5HgGs3j8B3l5VVwCXAP8yyQfaWKbW1anraFV9p6peAP5XVX0LoKr+H/DSdFs77fzatBs4xbwEvPEY9fM5Tf7b9BrA6r2mqv4PQFU9leQS4O4kP4ABMIoXk7y2BcDbXy4meQOnyf9kaynJw8sNAeetZS+ngV8E7kvyBHCo1d4EvBm4btmtTiGeAlqlJJ8Crq+qLwzV1gP7gJ+pqnVTa+4UlOTMqvr2MeobgPOr6pEptHXKSvJ14DLguaVDwP+sqmP9RatlJHkNg6+xH74I/GBVfWd6XU2ORwCrdzVwdLhQVUeBq5P8x+m0dOo61j/+rf4NTo2v3X21+QPg9cN/oLwsyWfWvp1TW1W9BNw/7T5OFo8AJKlTXgSWpE4ZAJLUKQNAkjplAEhSp/4KHEbKxlESGUEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 数据分析\r\n",
    "%cd ~\r\n",
    "import pandas as pd\r\n",
    "from matplotlib import pyplot as plt\r\n",
    "%matplotlib inline\r\n",
    "\r\n",
    "data=pd.read_csv('train_period.txt', header=None, sep=' ')\r\n",
    "print(data[1].value_counts())\r\n",
    "data[1].value_counts().plot(kind=\"bar\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train dataset len: 4160\n",
      "eval dataset len: 1040\n"
     ]
    }
   ],
   "source": [
    "# 训练集、测试集划分\r\n",
    "import pandas as pd\r\n",
    "import os\r\n",
    "from sklearn.model_selection import train_test_split\r\n",
    "\r\n",
    "def split_dataset(data_file):\r\n",
    "    # 展示不同的调用方式\r\n",
    "    data = pd.read_csv(data_file, header=None, sep=' ')\r\n",
    "    train_dataset, eval_dataset = train_test_split(data, test_size=0.2, random_state=42)\r\n",
    "    print(f'train dataset len: {train_dataset.size}')\r\n",
    "    print(f'eval dataset len: {eval_dataset.size}')\r\n",
    "    train_filename='train_' + data_file.split('.')[0]+'.txt'\r\n",
    "    eval_filename='eval_' + data_file.split('.')[0]+'.txt'\r\n",
    "    train_dataset.to_csv(train_filename, index=None, header=None, sep=' ')\r\n",
    "    eval_dataset.to_csv(eval_filename, index=None, header=None, sep=' ')\r\n",
    "    \r\n",
    "\r\n",
    "data_file='train_period.txt'\r\n",
    "split_dataset(data_file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# pip更新或安装包后需要重启notebook\r\n",
    "!pip install -U scikit-learn\r\n",
    "# 数据均衡用\r\n",
    "!pip install -U imblearn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "**************************************************\n",
      "2    1304\n",
      "3     653\n",
      "1      95\n",
      "0      28\n",
      "Name: 1, dtype: int64\n",
      "Counter({2: 1304, 3: 653, 1: 95, 0: 28})\n",
      "**************************************************\n",
      "Counter({2: 1304, 3: 1304, 1: 1304, 0: 1304})\n",
      "5216\n",
      "**************************************************\n",
      "5216\n",
      "**************************************************\n",
      "2    309\n",
      "3    176\n",
      "1     29\n",
      "0      6\n",
      "Name: 1, dtype: int64\n",
      "Counter({2: 309, 3: 176, 1: 29, 0: 6})\n",
      "**************************************************\n",
      "Counter({2: 309, 3: 309, 1: 309, 0: 309})\n",
      "1236\n",
      "**************************************************\n",
      "1236\n"
     ]
    }
   ],
   "source": [
    "# 数据均衡\r\n",
    "import pandas as pd\r\n",
    "from collections import Counter\r\n",
    "from imblearn.over_sampling import SMOTE\r\n",
    "import numpy as np\r\n",
    "\r\n",
    "def upsampleing(filename):\r\n",
    "    print(50 * '*')\r\n",
    "    data = pd.read_csv(filename, header=None, sep=' ')\r\n",
    "    print(data[1].value_counts())\r\n",
    "    # 查看各个标签的样本量\r\n",
    "    print(Counter(data[1]))\r\n",
    "    print(50 * '*')\r\n",
    "    # 数据均衡\r\n",
    "    X = np.array(data[0].index.tolist()).reshape(-1, 1)\r\n",
    "    y = data[1]\r\n",
    "    ros = SMOTE(random_state=0)\r\n",
    "    X_resampled, y_resampled = ros.fit_resample(X, y)\r\n",
    "    print(Counter(y_resampled))\r\n",
    "    print(len(y_resampled))\r\n",
    "    print(50 * '*')\r\n",
    "    img_list=[]\r\n",
    "    for i in range(len(X_resampled)):\r\n",
    "        img_list.append(data.loc[X_resampled[i]][0].tolist()[0])\r\n",
    "    dict_weather={'0':img_list, '1':y_resampled.values}\r\n",
    "    newdata=pd.DataFrame(dict_weather)\r\n",
    "    print(len(newdata))\r\n",
    "    new_filename=filename.split('.')[0]+'_imblearn'+'.txt'\r\n",
    "    newdata.to_csv(new_filename, header=None, index=None, sep=' ')\r\n",
    "    \r\n",
    "    \r\n",
    "\r\n",
    "filename='train_train_period.txt'\r\n",
    "upsampleing(filename)\r\n",
    "filename='eval_train_period.txt'\r\n",
    "upsampleing(filename)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 5.按天气分制作标签"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "写入train_weather.txt完成！！！\n"
     ]
    }
   ],
   "source": [
    "import json\r\n",
    "import os\r\n",
    "\r\n",
    "train = {}\r\n",
    "with open('train.json', 'r') as f:\r\n",
    "    train = json.load(f)\r\n",
    "\r\n",
    "weather_list =  {'Cloudy': 0, 'Rainy': 1, 'Sunny': 2}\r\n",
    "f_weather=open('train_weather.txt','w')\r\n",
    "for item in train[\"annotations\"]:\r\n",
    "    label = weather_list[item['weather']] \r\n",
    "    file_name=os.path.join(item['filename'].split('\\\\')[0], item['filename'].split('\\\\')[1])\r\n",
    "    f_weather.write(file_name +' '+ str(label) +'\\n')\r\n",
    "f_weather.close()\r\n",
    "print(\"写入train_weather.txt完成！！！\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 6.数据集划分并均衡"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0    1119\n",
      "2     886\n",
      "1     595\n",
      "Name: 1, dtype: int64\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7feffe82d190>"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD4CAYAAADlwTGnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAADQxJREFUeJzt3X+o3fV9x/HnSzPbtTJ/XoJNsl7BMHGDbe5iHcKQZrT+GEv+aMVSZpCw/GNbOwczGwzZ9o/CmFMYstDYRSiu4goJVVYkKmMMM69WbDXrvDhtEvxxW6NbkdJmvvfH/WQ9TZMmOd+bcxI/zwdc7vf7+X7O+X7ChTzz/Z5zblJVSJL6c8a0FyBJmg4DIEmdMgCS1CkDIEmdMgCS1CkDIEmdMgCS1CkDIEmdMgCS1KkV017Az3PhhRfW7OzstJchSaeVZ5555ntVNXOsead0AGZnZ5mfn5/2MiTptJLk1eOZ5y0gSeqUAZCkThkASeqUAZCkThkASeqUAZCkThkASeqUAZCkTp3SHwSbtNktj0x7CSfVK3deP+0lSDqFeAUgSZ0yAJLUKQMgSZ0yAJLUKQMgSZ0yAJLUKQMgSZ0yAJLUKQMgSZ0yAJLUKQMgSZ0yAJLUKQMgSZ0yAJLUKQMgSZ0yAJLUKQMgSZ0yAJLUqWMGIMn9Sd5M8u2RsfOTPJbkpfb9vDaeJPcmWUjyfJLLRx6zsc1/KcnGk/PHkSQdr+O5AvgH4JrDxrYAu6pqLbCr7QNcC6xtX5uB+2ApGMAdwMeAK4A7DkVDkjQdxwxAVf0L8NZhw+uB7W17O7BhZPyBWvIUcG6Si4BPAo9V1VtVdQB4jJ+NiiRpgsZ9DWBlVb3Wtl8HVrbtVcDekXn72tjRxn9Gks1J5pPMLy4ujrk8SdKxDH4RuKoKqGVYy6Hn21pVc1U1NzMzs1xPK0k6zLgBeKPd2qF9f7ON7wfWjMxb3caONi5JmpJxA7ATOPROno3AjpHxm9q7ga4E3mm3ir4BfCLJee3F30+0MUnSlKw41oQkDwJXAxcm2cfSu3nuBB5Ksgl4FbihTX8UuA5YAN4FbgaoqreS/BXwdJv3l1V1+AvLkqQJOmYAquozRzm07ghzC7jlKM9zP3D/Ca1OknTS+ElgSeqUAZCkThkASeqUAZCkThkASeqUAZCkThkASeqUAZCkThkASeqUAZCkThkASeqUAZCkThkASeqUAZCkThkASerUMf8/AOl0MbvlkWkv4aR65c7rp70Evc94BSBJnTIAktQpAyBJnTIAktQpAyBJnTIAktQpAyBJnTIAktQpAyBJnTIAktQpAyBJnTIAktSpQQFI8kdJXkjy7SQPJvlgkouT7E6ykOSrSc5qcz/Q9hfa8dnl+ANIksYzdgCSrAK+AMxV1a8BZwI3AncBd1fVJcABYFN7yCbgQBu/u82TJE3J0FtAK4BfTLIC+BDwGvBx4OF2fDuwoW2vb/u04+uSZOD5JUljGjsAVbUf+Gvguyz9xf8O8AzwdlUdbNP2Aava9ipgb3vswTb/gsOfN8nmJPNJ5hcXF8ddniTpGIbcAjqPpX/VXwx8BPgwcM3QBVXV1qqaq6q5mZmZoU8nSTqKIbeAfhf4r6parKofA18DrgLObbeEAFYD+9v2fmANQDt+DvD9AeeXJA0wJADfBa5M8qF2L38d8CLwBPCpNmcjsKNt72z7tOOPV1UNOL8kaYAhrwHsZunF3GeBb7Xn2grcDtyWZIGle/zb2kO2ARe08duALQPWLUkaaNB/Cl9VdwB3HDb8MnDFEeb+EPj0kPNJkpaPnwSWpE4ZAEnqlAGQpE4ZAEnqlAGQpE4ZAEnqlAGQpE4ZAEnqlAGQpE4ZAEnqlAGQpE4ZAEnqlAGQpE4ZAEnqlAGQpE4ZAEnqlAGQpE4ZAEnqlAGQpE4ZAEnqlAGQpE4ZAEnqlAGQpE4ZAEnqlAGQpE4ZAEnqlAGQpE4ZAEnq1KAAJDk3ycNJ/iPJniS/neT8JI8leal9P6/NTZJ7kywkeT7J5cvzR5AkjWPoFcA9wD9X1aXArwN7gC3ArqpaC+xq+wDXAmvb12bgvoHnliQNMHYAkpwD/A6wDaCqflRVbwPrge1t2nZgQ9teDzxQS54Czk1y0dgrlyQNMuQK4GJgEfhykm8m+VKSDwMrq+q1Nud1YGXbXgXsHXn8vjb2U5JsTjKfZH5xcXHA8iRJP8+KgY+9HPh8Ve1Ocg8/ud0DQFVVkjqRJ62qrcBWgLm5uRN6rKTT0+yWR6a9hJPqlTuvn/YSjmjIFcA+YF9V7W77D7MUhDcO3dpp399sx/cDa0Yev7qNSZKmYOwAVNXrwN4kv9KG1gEvAjuBjW1sI7Cjbe8EbmrvBroSeGfkVpEkacKG3AIC+DzwlSRnAS8DN7MUlYeSbAJeBW5ocx8FrgMWgHfbXEnSlAwKQFU9B8wd4dC6I8wt4JYh55MkLR8/CSxJnTIAktQpAyBJnTIAktQpAyBJnTIAktQpAyBJnTIAktQpAyBJnTIAktQpAyBJnTIAktQpAyBJnTIAktQpAyBJnTIAktQpAyBJnTIAktQpAyBJnTIAktQpAyBJnTIAktQpAyBJnTIAktQpAyBJnTIAktQpAyBJnTIAktSpwQFIcmaSbyb5etu/OMnuJAtJvprkrDb+gba/0I7PDj23JGl8y3EFcCuwZ2T/LuDuqroEOABsauObgANt/O42T5I0JYMCkGQ1cD3wpbYf4OPAw23KdmBD217f9mnH17X5kqQpGHoF8LfAnwDvtf0LgLer6mDb3wesaturgL0A7fg7bf5PSbI5yXyS+cXFxYHLkyQdzdgBSPJ7wJtV9cwyroeq2lpVc1U1NzMzs5xPLUkasWLAY68Cfj/JdcAHgV8C7gHOTbKi/St/NbC/zd8PrAH2JVkBnAN8f8D5JUkDjH0FUFV/WlWrq2oWuBF4vKo+CzwBfKpN2wjsaNs72z7t+ONVVeOeX5I0zMn4HMDtwG1JFli6x7+tjW8DLmjjtwFbTsK5JUnHacgtoP9XVU8CT7btl4ErjjDnh8Cnl+N8kqTh/CSwJHXKAEhSpwyAJHXKAEhSpwyAJHXKAEhSpwyAJHXKAEhSpwyAJHXKAEhSpwyAJHXKAEhSpwyAJHXKAEhSpwyAJHXKAEhSpwyAJHXKAEhSpwyAJHXKAEhSpwyAJHXKAEhSpwyAJHXKAEhSpwyAJHXKAEhSpwyAJHXKAEhSp8YOQJI1SZ5I8mKSF5Lc2sbPT/JYkpfa9/PaeJLcm2QhyfNJLl+uP4Qk6cQNuQI4CPxxVV0GXAnckuQyYAuwq6rWArvaPsC1wNr2tRm4b8C5JUkDjR2Aqnqtqp5t2/8D7AFWAeuB7W3admBD214PPFBLngLOTXLR2CuXJA2yLK8BJJkFfhPYDaysqtfaodeBlW17FbB35GH72tjhz7U5yXyS+cXFxeVYniTpCAYHIMnZwD8BX6yq/x49VlUF1Ik8X1Vtraq5qpqbmZkZujxJ0lEMCkCSX2DpL/+vVNXX2vAbh27ttO9vtvH9wJqRh69uY5KkKRjyLqAA24A9VfU3I4d2Ahvb9kZgx8j4Te3dQFcC74zcKpIkTdiKAY+9CvgD4FtJnmtjfwbcCTyUZBPwKnBDO/YocB2wALwL3Dzg3JKkgcYOQFX9K5CjHF53hPkF3DLu+SRJy8tPAktSpwyAJHXKAEhSpwyAJHXKAEhSpwyAJHXKAEhSpwyAJHXKAEhSpwyAJHXKAEhSpwyAJHXKAEhSpwyAJHXKAEhSpwyAJHXKAEhSpwyAJHXKAEhSpwyAJHXKAEhSpwyAJHXKAEhSpwyAJHXKAEhSpwyAJHXKAEhSpwyAJHVq4gFIck2S7yRZSLJl0ueXJC2ZaACSnAn8HXAtcBnwmSSXTXINkqQlk74CuAJYqKqXq+pHwD8C6ye8BkkSsGLC51sF7B3Z3wd8bHRCks3A5rb7gyTfmdDapuFC4HuTOlnumtSZuuHP7/T1fv/ZffR4Jk06AMdUVVuBrdNexyQkma+quWmvQ+Px53f68me3ZNK3gPYDa0b2V7cxSdKETToATwNrk1yc5CzgRmDnhNcgSWLCt4Cq6mCSzwHfAM4E7q+qFya5hlNMF7e63sf8+Z2+/NkBqappr0GSNAV+EliSOmUAJKlTBkCSOnXKfQ7g/SzJpSx98nlVG9oP7KyqPdNblaReeQUwIUluZ+lXXwT49/YV4EF/Kd6pL8mlSdYlOfuw8WumtSZpKN8FNCFJ/hP41ar68WHjZwEvVNXa6axMx5LkC8AtwB7gN4Bbq2pHO/ZsVV0+zfVpfElurqovT3sd0+IVwOS8B3zkCOMXtWM6df0h8FtVtQG4GvjzJLe2Y5naqrQc/mLaC5gmXwOYnC8Cu5K8xE9+Id4vA5cAn5vaqnQ8zqiqHwBU1StJrgYeTvJRDMApL8nzRzsErJzkWk413gKaoCRnsPQrsUdfBH66qv53eqvSsSR5HLitqp4bGVsB3A98tqrOnNridExJ3gA+CRw4/BDwb1V1pCvzLngFMEFV9R7w1LTXoRN2E3BwdKCqDgI3Jfn76SxJJ+DrwNmjAT8kyZOTX86pwysASeqULwJLUqcMgCR1ygBIUqcMgCR16v8Aia3uG2/KDZoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import pandas as pd\r\n",
    "from matplotlib import pyplot as plt\r\n",
    "\r\n",
    "data=pd.read_csv('train_weather.txt', header=None, sep=' ')\r\n",
    "print(data[1].value_counts())\r\n",
    "data[1].value_counts().plot(kind=\"bar\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train dataset len: 4160\n",
      "eval dataset len: 1040\n"
     ]
    }
   ],
   "source": [
    "# 训练集、测试集划分\r\n",
    "\r\n",
    "import pandas as pd\r\n",
    "import os\r\n",
    "from sklearn.model_selection import train_test_split\r\n",
    "\r\n",
    "def split_dataset(data_file):\r\n",
    "    # 展示不同的调用方式\r\n",
    "    data = pd.read_csv(data_file, header=None, sep=' ')\r\n",
    "    train_dataset, eval_dataset = train_test_split(data, test_size=0.2, random_state=42)\r\n",
    "    print(f'train dataset len: {train_dataset.size}')\r\n",
    "    print(f'eval dataset len: {eval_dataset.size}')\r\n",
    "    train_filename='train_' + data_file.split('.')[0]+'.txt'\r\n",
    "    eval_filename='eval_' + data_file.split('.')[0]+'.txt'\r\n",
    "    train_dataset.to_csv(train_filename, index=None, header=None, sep=' ')\r\n",
    "    eval_dataset.to_csv(eval_filename, index=None, header=None, sep=' ')\r\n",
    "    \r\n",
    "\r\n",
    "data_file='train_weather.txt'\r\n",
    "split_dataset(data_file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "**************************************************\n",
      "0    892\n",
      "2    715\n",
      "1    473\n",
      "Name: 1, dtype: int64\n",
      "Counter({0: 892, 2: 715, 1: 473})\n",
      "**************************************************\n",
      "Counter({0: 892, 2: 892, 1: 892})\n",
      "2676\n",
      "**************************************************\n",
      "2676\n",
      "**************************************************\n",
      "0    227\n",
      "2    171\n",
      "1    122\n",
      "Name: 1, dtype: int64\n",
      "Counter({0: 227, 2: 171, 1: 122})\n",
      "**************************************************\n",
      "Counter({0: 227, 2: 227, 1: 227})\n",
      "681\n",
      "**************************************************\n",
      "681\n"
     ]
    }
   ],
   "source": [
    "# 数据均衡\r\n",
    "import pandas as pd\r\n",
    "from collections import Counter\r\n",
    "from imblearn.over_sampling import SMOTE\r\n",
    "import numpy as np\r\n",
    "\r\n",
    "def upsampleing(filename):\r\n",
    "    print(50 * '*')\r\n",
    "    data = pd.read_csv(filename, header=None, sep=' ')\r\n",
    "    print(data[1].value_counts())\r\n",
    "    # 查看各个标签的样本量\r\n",
    "    print(Counter(data[1]))\r\n",
    "    print(50 * '*')\r\n",
    "    # 数据均衡\r\n",
    "    X = np.array(data[0].index.tolist()).reshape(-1, 1)\r\n",
    "    y = data[1]\r\n",
    "    ros = SMOTE(random_state=0)\r\n",
    "    X_resampled, y_resampled = ros.fit_resample(X, y)\r\n",
    "    print(Counter(y_resampled))\r\n",
    "    print(len(y_resampled))\r\n",
    "    print(50 * '*')\r\n",
    "    img_list=[]\r\n",
    "    for i in range(len(X_resampled)):\r\n",
    "        img_list.append(data.loc[X_resampled[i]][0].tolist()[0])\r\n",
    "    dict_weather={'0':img_list, '1':y_resampled.values}\r\n",
    "    newdata=pd.DataFrame(dict_weather)\r\n",
    "    print(len(newdata))\r\n",
    "    new_filename=filename.split('.')[0]+'_imblearn'+'.txt'\r\n",
    "    newdata.to_csv(new_filename, header=None, index=None, sep=' ')\r\n",
    "    \r\n",
    "    \r\n",
    "\r\n",
    "filename='train_train_weather.txt'\r\n",
    "upsampleing(filename)\r\n",
    "filename='eval_train_weather.txt'\r\n",
    "upsampleing(filename)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 三、环境准备\n",
    "飞桨图像识别套件PaddleClas是飞桨为工业界和学术界所准备的一个图像识别任务的工具集，助力使用者训练出更好的视觉模型和应用落地。此次计划使用端到端的PaddleClas图像分类套件来快速完成分类。此次使用PaddleClas框架完成比赛。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "fatal: destination path 'PaddleClas' already exists and is not an empty directory.\r\n"
     ]
    }
   ],
   "source": [
    "# git 下载PaddleClas\r\n",
    "!git clone https://gitee.com/paddlepaddle/PaddleClas.git --depth=1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 安装\r\n",
    "%cd ~/PaddleClas/\r\n",
    "!pip install -U pip\r\n",
    "!pip install -r requirements.txt\r\n",
    "!pip install -e ./\r\n",
    "%cd ~"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 四、模型训练 and 评估"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 1.时间训练\n",
    "以 **PaddleClas/ppcls/configs/ImageNet/VisionTransformer/ViT_small_patch16_224.yaml** 为基础进行修改\n",
    "\n",
    "```\n",
    "# global configs\n",
    "Global:\n",
    "  checkpoints: null\n",
    "  pretrained_model: null\n",
    "  output_dir: ./output/\n",
    "  device: gpu\n",
    "  save_interval: 1\n",
    "  eval_during_train: True\n",
    "  eval_interval: 1\n",
    "  epochs: 120\n",
    "  print_batch_step: 10\n",
    "  use_visualdl: False\n",
    "  # used for static mode and model export\n",
    "  image_shape: [3, 224, 224]\n",
    "  save_inference_dir: ./inference\n",
    "\n",
    "# model architecture\n",
    "Arch:\n",
    "  name: ViT_small_patch16_224\n",
    "  class_num: 1000\n",
    " \n",
    "# loss function config for traing/eval process\n",
    "Loss:\n",
    "  Train:\n",
    "    - CELoss:\n",
    "        weight: 1.0\n",
    "  Eval:\n",
    "    - CELoss:\n",
    "        weight: 1.0\n",
    "\n",
    "\n",
    "Optimizer:\n",
    "  name: Momentum\n",
    "  momentum: 0.9\n",
    "  lr:\n",
    "    name: Piecewise\n",
    "    learning_rate: 0.1\n",
    "    decay_epochs: [30, 60, 90]\n",
    "    values: [0.1, 0.01, 0.001, 0.0001]\n",
    "  regularizer:\n",
    "    name: 'L2'\n",
    "    coeff: 0.0001\n",
    "\n",
    "\n",
    "# data loader for train and eval\n",
    "DataLoader:\n",
    "  Train:\n",
    "    dataset:\n",
    "      name: ImageNetDataset\n",
    "      image_root: ./dataset/ILSVRC2012/\n",
    "      cls_label_path: ./dataset/ILSVRC2012/train_list.txt\n",
    "      transform_ops:\n",
    "        - DecodeImage:\n",
    "            to_rgb: True\n",
    "            channel_first: False\n",
    "        - RandCropImage:\n",
    "            size: 224\n",
    "        - RandFlipImage:\n",
    "            flip_code: 1\n",
    "        - NormalizeImage:\n",
    "            scale: 1.0/255.0\n",
    "            mean: [0.5, 0.5, 0.5]\n",
    "            std: [0.5, 0.5, 0.5]\n",
    "            order: ''\n",
    "\n",
    "    sampler:\n",
    "      name: DistributedBatchSampler\n",
    "      batch_size: 64\n",
    "      drop_last: False\n",
    "      shuffle: True\n",
    "    loader:\n",
    "      num_workers: 4\n",
    "      use_shared_memory: True\n",
    "\n",
    "  Eval:\n",
    "    dataset: \n",
    "      name: ImageNetDataset\n",
    "      image_root: ./dataset/ILSVRC2012/\n",
    "      cls_label_path: ./dataset/ILSVRC2012/val_list.txt\n",
    "      transform_ops:\n",
    "        - DecodeImage:\n",
    "            to_rgb: True\n",
    "            channel_first: False\n",
    "        - ResizeImage:\n",
    "            resize_short: 256\n",
    "        - CropImage:\n",
    "            size: 224\n",
    "        - NormalizeImage:\n",
    "            scale: 1.0/255.0\n",
    "            mean: [0.5, 0.5, 0.5]\n",
    "            std: [0.5, 0.5, 0.5]\n",
    "            order: ''\n",
    "    sampler:\n",
    "      name: DistributedBatchSampler\n",
    "      batch_size: 64\n",
    "      drop_last: False\n",
    "      shuffle: False\n",
    "    loader:\n",
    "      num_workers: 4\n",
    "      use_shared_memory: True\n",
    "\n",
    "Infer:\n",
    "  infer_imgs: docs/images/whl/demo.jpg\n",
    "  batch_size: 10\n",
    "  transforms:\n",
    "    - DecodeImage:\n",
    "        to_rgb: True\n",
    "        channel_first: False\n",
    "    - ResizeImage:\n",
    "        resize_short: 256\n",
    "    - CropImage:\n",
    "        size: 224\n",
    "    - NormalizeImage:\n",
    "        scale: 1.0/255.0\n",
    "        mean: [0.5, 0.5, 0.5]\n",
    "        std: [0.5, 0.5, 0.5]\n",
    "        order: ''\n",
    "    - ToCHWImage:\n",
    "  PostProcess:\n",
    "    name: Topk\n",
    "    topk: 5\n",
    "    class_id_map_file: ppcls/utils/imagenet1k_label_list.txt\n",
    "\n",
    "Metric:\n",
    "  Train:\n",
    "    - TopkAcc:\n",
    "        topk: [1, 5]\n",
    "  Eval:\n",
    "    - TopkAcc:\n",
    "        topk: [1, 5]\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/aistudio\n"
     ]
    }
   ],
   "source": [
    "# 覆盖配置\r\n",
    "%cd ~\r\n",
    "!cp -f ~/ViT_small_patch16_224.yaml ~/ppcls/configs/ImageNet/VisionTransformer/ViT_small_patch16_224.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/aistudio/PaddleClas\n"
     ]
    }
   ],
   "source": [
    "# 开始训练\r\n",
    "%cd ~/PaddleClas/\r\n",
    "\r\n",
    "!python3 tools/train.py \\\r\n",
    "    -c ./ppcls/configs/ImageNet/VisionTransformer/ViT_base_patch16_224.yaml \\\r\n",
    "    -o Arch.pretrained=True \\\r\n",
    "    -o Global.pretrained_model=./output/ViT_base_patch16_224/epoch_21 \\\r\n",
    "    -o Global.device=gpu"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/aistudio/PaddleClas\n",
      "[2022/01/18 00:16:10] root INFO: \n",
      "===========================================================\n",
      "==        PaddleClas is powered by PaddlePaddle !        ==\n",
      "===========================================================\n",
      "==                                                       ==\n",
      "==   For more info please go to the following website.   ==\n",
      "==                                                       ==\n",
      "==       https://github.com/PaddlePaddle/PaddleClas      ==\n",
      "===========================================================\n",
      "\n",
      "[2022/01/18 00:16:10] root INFO: Arch : \n",
      "[2022/01/18 00:16:10] root INFO:     class_num : 4\n",
      "[2022/01/18 00:16:10] root INFO:     name : ViT_base_patch16_224\n",
      "[2022/01/18 00:16:10] root INFO: DataLoader : \n",
      "[2022/01/18 00:16:10] root INFO:     Eval : \n",
      "[2022/01/18 00:16:10] root INFO:         dataset : \n",
      "[2022/01/18 00:16:10] root INFO:             cls_label_path : /home/aistudio/eval_train_period_imblearn.txt\n",
      "[2022/01/18 00:16:10] root INFO:             image_root : /home/aistudio/\n",
      "[2022/01/18 00:16:10] root INFO:             name : ImageNetDataset\n",
      "[2022/01/18 00:16:10] root INFO:             transform_ops : \n",
      "[2022/01/18 00:16:10] root INFO:                 DecodeImage : \n",
      "[2022/01/18 00:16:10] root INFO:                     channel_first : False\n",
      "[2022/01/18 00:16:10] root INFO:                     to_rgb : True\n",
      "[2022/01/18 00:16:10] root INFO:                 ResizeImage : \n",
      "[2022/01/18 00:16:10] root INFO:                     resize_short : 256\n",
      "[2022/01/18 00:16:10] root INFO:                 CropImage : \n",
      "[2022/01/18 00:16:10] root INFO:                     size : 224\n",
      "[2022/01/18 00:16:10] root INFO:                 NormalizeImage : \n",
      "[2022/01/18 00:16:10] root INFO:                     mean : [0.5, 0.5, 0.5]\n",
      "[2022/01/18 00:16:10] root INFO:                     order : \n",
      "[2022/01/18 00:16:10] root INFO:                     scale : 1.0/255.0\n",
      "[2022/01/18 00:16:10] root INFO:                     std : [0.5, 0.5, 0.5]\n",
      "[2022/01/18 00:16:10] root INFO:         loader : \n",
      "[2022/01/18 00:16:10] root INFO:             num_workers : 4\n",
      "[2022/01/18 00:16:10] root INFO:             use_shared_memory : True\n",
      "[2022/01/18 00:16:10] root INFO:         sampler : \n",
      "[2022/01/18 00:16:10] root INFO:             batch_size : 128\n",
      "[2022/01/18 00:16:10] root INFO:             drop_last : False\n",
      "[2022/01/18 00:16:10] root INFO:             name : DistributedBatchSampler\n",
      "[2022/01/18 00:16:10] root INFO:             shuffle : False\n",
      "[2022/01/18 00:16:10] root INFO:     Train : \n",
      "[2022/01/18 00:16:10] root INFO:         dataset : \n",
      "[2022/01/18 00:16:10] root INFO:             cls_label_path : /home/aistudio/train_train_period_imblearn.txt\n",
      "[2022/01/18 00:16:10] root INFO:             image_root : /home/aistudio\n",
      "[2022/01/18 00:16:10] root INFO:             name : ImageNetDataset\n",
      "[2022/01/18 00:16:10] root INFO:             transform_ops : \n",
      "[2022/01/18 00:16:10] root INFO:                 DecodeImage : \n",
      "[2022/01/18 00:16:10] root INFO:                     channel_first : False\n",
      "[2022/01/18 00:16:10] root INFO:                     to_rgb : True\n",
      "[2022/01/18 00:16:10] root INFO:                 RandCropImage : \n",
      "[2022/01/18 00:16:10] root INFO:                     size : 224\n",
      "[2022/01/18 00:16:10] root INFO:                 RandFlipImage : \n",
      "[2022/01/18 00:16:10] root INFO:                     flip_code : 1\n",
      "[2022/01/18 00:16:10] root INFO:                 NormalizeImage : \n",
      "[2022/01/18 00:16:10] root INFO:                     mean : [0.5, 0.5, 0.5]\n",
      "[2022/01/18 00:16:10] root INFO:                     order : \n",
      "[2022/01/18 00:16:10] root INFO:                     scale : 1.0/255.0\n",
      "[2022/01/18 00:16:10] root INFO:                     std : [0.5, 0.5, 0.5]\n",
      "[2022/01/18 00:16:10] root INFO:         loader : \n",
      "[2022/01/18 00:16:10] root INFO:             num_workers : 4\n",
      "[2022/01/18 00:16:10] root INFO:             use_shared_memory : True\n",
      "[2022/01/18 00:16:10] root INFO:         sampler : \n",
      "[2022/01/18 00:16:10] root INFO:             batch_size : 160\n",
      "[2022/01/18 00:16:10] root INFO:             drop_last : False\n",
      "[2022/01/18 00:16:10] root INFO:             name : DistributedBatchSampler\n",
      "[2022/01/18 00:16:10] root INFO:             shuffle : True\n",
      "[2022/01/18 00:16:10] root INFO: Global : \n",
      "[2022/01/18 00:16:10] root INFO:     checkpoints : None\n",
      "[2022/01/18 00:16:10] root INFO:     device : gpu\n",
      "[2022/01/18 00:16:10] root INFO:     epochs : 120\n",
      "[2022/01/18 00:16:10] root INFO:     eval_during_train : True\n",
      "[2022/01/18 00:16:10] root INFO:     eval_interval : 1\n",
      "[2022/01/18 00:16:10] root INFO:     image_shape : [3, 224, 224]\n",
      "[2022/01/18 00:16:10] root INFO:     output_dir : ./output/\n",
      "[2022/01/18 00:16:10] root INFO:     pretrained_model : ./output/ViT_base_patch16_224/best_model\n",
      "[2022/01/18 00:16:10] root INFO:     print_batch_step : 10\n",
      "[2022/01/18 00:16:10] root INFO:     save_inference_dir : ./inference\n",
      "[2022/01/18 00:16:10] root INFO:     save_interval : 1\n",
      "[2022/01/18 00:16:10] root INFO:     use_visualdl : False\n",
      "[2022/01/18 00:16:10] root INFO: Infer : \n",
      "[2022/01/18 00:16:10] root INFO:     PostProcess : \n",
      "[2022/01/18 00:16:10] root INFO:         class_id_map_file : ppcls/utils/imagenet1k_label_list.txt\n",
      "[2022/01/18 00:16:10] root INFO:         name : Topk\n",
      "[2022/01/18 00:16:10] root INFO:         topk : 5\n",
      "[2022/01/18 00:16:10] root INFO:     batch_size : 10\n",
      "[2022/01/18 00:16:10] root INFO:     infer_imgs : docs/images/whl/demo.jpg\n",
      "[2022/01/18 00:16:10] root INFO:     transforms : \n",
      "[2022/01/18 00:16:10] root INFO:         DecodeImage : \n",
      "[2022/01/18 00:16:10] root INFO:             channel_first : False\n",
      "[2022/01/18 00:16:10] root INFO:             to_rgb : True\n",
      "[2022/01/18 00:16:10] root INFO:         ResizeImage : \n",
      "[2022/01/18 00:16:10] root INFO:             resize_short : 256\n",
      "[2022/01/18 00:16:10] root INFO:         CropImage : \n",
      "[2022/01/18 00:16:10] root INFO:             size : 224\n",
      "[2022/01/18 00:16:10] root INFO:         NormalizeImage : \n",
      "[2022/01/18 00:16:10] root INFO:             mean : [0.5, 0.5, 0.5]\n",
      "[2022/01/18 00:16:10] root INFO:             order : \n",
      "[2022/01/18 00:16:10] root INFO:             scale : 1.0/255.0\n",
      "[2022/01/18 00:16:10] root INFO:             std : [0.5, 0.5, 0.5]\n",
      "[2022/01/18 00:16:10] root INFO:         ToCHWImage : None\n",
      "[2022/01/18 00:16:10] root INFO: Loss : \n",
      "[2022/01/18 00:16:10] root INFO:     Eval : \n",
      "[2022/01/18 00:16:10] root INFO:         CELoss : \n",
      "[2022/01/18 00:16:10] root INFO:             weight : 1.0\n",
      "[2022/01/18 00:16:10] root INFO:     Train : \n",
      "[2022/01/18 00:16:10] root INFO:         CELoss : \n",
      "[2022/01/18 00:16:10] root INFO:             weight : 1.0\n",
      "[2022/01/18 00:16:10] root INFO: Metric : \n",
      "[2022/01/18 00:16:10] root INFO:     Eval : \n",
      "[2022/01/18 00:16:10] root INFO:         TopkAcc : \n",
      "[2022/01/18 00:16:10] root INFO:             topk : [1, 2]\n",
      "[2022/01/18 00:16:10] root INFO:     Train : \n",
      "[2022/01/18 00:16:10] root INFO:         TopkAcc : \n",
      "[2022/01/18 00:16:10] root INFO:             topk : [1, 2]\n",
      "[2022/01/18 00:16:10] root INFO: Optimizer : \n",
      "[2022/01/18 00:16:10] root INFO:     lr : \n",
      "[2022/01/18 00:16:10] root INFO:         decay_epochs : [10, 22, 30]\n",
      "[2022/01/18 00:16:10] root INFO:         learning_rate : 0.01\n",
      "[2022/01/18 00:16:10] root INFO:         name : Piecewise\n",
      "[2022/01/18 00:16:10] root INFO:         values : [0.01, 0.001, 0.0001, 1e-05]\n",
      "[2022/01/18 00:16:10] root INFO:     momentum : 0.9\n",
      "[2022/01/18 00:16:10] root INFO:     name : Momentum\n",
      "[2022/01/18 00:16:10] root INFO:     regularizer : \n",
      "[2022/01/18 00:16:10] root INFO:         coeff : 0.0001\n",
      "[2022/01/18 00:16:10] root INFO:         name : L2\n",
      "[2022/01/18 00:16:10] root INFO: train with paddle 2.2.1 and device CUDAPlace(0)\n",
      "W0118 00:16:10.219051  1924 device_context.cc:447] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 10.1, Runtime API Version: 10.1\n",
      "W0118 00:16:10.224006  1924 device_context.cc:465] device: 0, cuDNN Version: 7.6.\n",
      "[2022/01/18 00:16:21] root INFO: [Eval][Epoch 0][Iter: 0/10]CELoss: 0.68425, loss: 0.68425, top1: 0.79688, top2: 0.92969, batch_cost: 6.03284s, reader_cost: 5.55918, ips: 21.21719 images/sec\n",
      "[2022/01/18 00:16:30] root INFO: [Eval][Epoch 0][Avg]CELoss: 1.24542, loss: 1.24542, top1: 0.47006, top2: 0.73463\n"
     ]
    }
   ],
   "source": [
    "# 模型评估\r\n",
    "%cd ~/PaddleClas/\r\n",
    "\r\n",
    "!python  tools/eval.py \\\r\n",
    "        -c  ./ppcls/configs/ImageNet/VisionTransformer/ViT_base_patch16_224.yaml \\\r\n",
    "        -o Global.pretrained_model=./output/ViT_base_patch16_224/best_model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 2.天气训练\n",
    "\n",
    "配置文件为：** PaddleClas/ppcls/configs/ImageNet/VisionTransformer/ViT_base_patch16_224_weather.yaml**\n",
    "```\n",
    "# global configs\n",
    "Global:\n",
    "  checkpoints: null\n",
    "  pretrained_model: null\n",
    "  output_dir: ./output_weather/\n",
    "  device: gpu\n",
    "  save_interval: 1\n",
    "  eval_during_train: True\n",
    "  eval_interval: 1\n",
    "  epochs: 120\n",
    "  print_batch_step: 10\n",
    "  use_visualdl: False\n",
    "  # used for static mode and model export\n",
    "  image_shape: [3, 224, 224]\n",
    "  save_inference_dir: ./inference_weather\n",
    "\n",
    "# model architecture\n",
    "Arch:\n",
    "  name: ViT_base_patch16_224\n",
    "  class_num: 3\n",
    " \n",
    "# loss function config for traing/eval process\n",
    "Loss:\n",
    "  Train:\n",
    "    - CELoss:\n",
    "        weight: 1.0\n",
    "  Eval:\n",
    "    - CELoss:\n",
    "        weight: 1.0\n",
    "\n",
    "\n",
    "Optimizer:\n",
    "  name: Momentum\n",
    "  momentum: 0.9\n",
    "  lr:\n",
    "    name: Piecewise\n",
    "    learning_rate: 0.01\n",
    "    decay_epochs: [10, 22, 30]\n",
    "    values: [0.01, 0.001, 0.0001, 0.00001]\n",
    "  regularizer:\n",
    "    name: 'L2'\n",
    "    coeff: 0.0001\n",
    "\n",
    "\n",
    "# data loader for train and eval\n",
    "DataLoader:\n",
    "  Train:\n",
    "    dataset:\n",
    "      name: ImageNetDataset\n",
    "      image_root: /home/aistudio\n",
    "      cls_label_path: /home/aistudio/train_train_weather_imblearn.txt\n",
    "      transform_ops:\n",
    "        - DecodeImage:\n",
    "            to_rgb: True\n",
    "            channel_first: False\n",
    "        - RandCropImage:\n",
    "            size: 224\n",
    "        - RandFlipImage:\n",
    "            flip_code: 1\n",
    "        - NormalizeImage:\n",
    "            scale: 1.0/255.0\n",
    "            mean: [0.5, 0.5, 0.5]\n",
    "            std: [0.5, 0.5, 0.5]\n",
    "            order: ''\n",
    "\n",
    "    sampler:\n",
    "      name: DistributedBatchSampler\n",
    "      batch_size: 160\n",
    "      drop_last: False\n",
    "      shuffle: True\n",
    "    loader:\n",
    "      num_workers: 4\n",
    "      use_shared_memory: True\n",
    "\n",
    "  Eval:\n",
    "    dataset: \n",
    "      name: ImageNetDataset\n",
    "      image_root: /home/aistudio/\n",
    "      cls_label_path: /home/aistudio/eval_train_weather_imblearn.txt\n",
    "      transform_ops:\n",
    "        - DecodeImage:\n",
    "            to_rgb: True\n",
    "            channel_first: False\n",
    "        - ResizeImage:\n",
    "            resize_short: 256\n",
    "        - CropImage:\n",
    "            size: 224\n",
    "        - NormalizeImage:\n",
    "            scale: 1.0/255.0\n",
    "            mean: [0.5, 0.5, 0.5]\n",
    "            std: [0.5, 0.5, 0.5]\n",
    "            order: ''\n",
    "    sampler:\n",
    "      name: DistributedBatchSampler\n",
    "      batch_size: 128\n",
    "      drop_last: False\n",
    "      shuffle: False\n",
    "    loader:\n",
    "      num_workers: 4\n",
    "      use_shared_memory: True\n",
    "\n",
    "Infer:\n",
    "  infer_imgs: docs/images/whl/demo.jpg\n",
    "  batch_size: 10\n",
    "  transforms:\n",
    "    - DecodeImage:\n",
    "        to_rgb: True\n",
    "        channel_first: False\n",
    "    - ResizeImage:\n",
    "        resize_short: 256\n",
    "    - CropImage:\n",
    "        size: 224\n",
    "    - NormalizeImage:\n",
    "        scale: 1.0/255.0\n",
    "        mean: [0.5, 0.5, 0.5]\n",
    "        std: [0.5, 0.5, 0.5]\n",
    "        order: ''\n",
    "    - ToCHWImage:\n",
    "  PostProcess:\n",
    "    name: Topk\n",
    "    topk: 5\n",
    "    class_id_map_file: ppcls/utils/imagenet1k_label_list.txt\n",
    "\n",
    "Metric:\n",
    "  Train:\n",
    "    - TopkAcc:\n",
    "        topk: [1, 2]\n",
    "  Eval:\n",
    "    - TopkAcc:\n",
    "        topk: [1, 2]\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/aistudio\n",
      "cp: cannot stat 'PaddleClas/ppcls/configs/ImageNet/VisionTransformer/ViT_small_patch16_224_weather.yaml': No such file or directory\r\n"
     ]
    }
   ],
   "source": [
    "# 覆盖配置\r\n",
    "%cd ~\r\n",
    "!cp -f  ~/ViT_small_patch16_224_weather.yaml ~/ppcls/configs/ImageNet/VisionTransformer/ViT_small_patch16_224_weather.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 模型训练\r\n",
    "%cd ~/PaddleClas/\r\n",
    "\r\n",
    "!python3 tools/train.py \\\r\n",
    "    -c ./ppcls/configs/ImageNet/VisionTransformer/ViT_base_patch16_224_weather.yaml \\\r\n",
    "    -o Arch.pretrained=True \\\r\n",
    "    -o Global.device=gpu"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/aistudio/PaddleClas\n",
      "[2022/01/18 00:17:24] root INFO: \n",
      "===========================================================\n",
      "==        PaddleClas is powered by PaddlePaddle !        ==\n",
      "===========================================================\n",
      "==                                                       ==\n",
      "==   For more info please go to the following website.   ==\n",
      "==                                                       ==\n",
      "==       https://github.com/PaddlePaddle/PaddleClas      ==\n",
      "===========================================================\n",
      "\n",
      "[2022/01/18 00:17:24] root INFO: Arch : \n",
      "[2022/01/18 00:17:24] root INFO:     class_num : 3\n",
      "[2022/01/18 00:17:24] root INFO:     name : ViT_base_patch16_224\n",
      "[2022/01/18 00:17:24] root INFO: DataLoader : \n",
      "[2022/01/18 00:17:24] root INFO:     Eval : \n",
      "[2022/01/18 00:17:24] root INFO:         dataset : \n",
      "[2022/01/18 00:17:24] root INFO:             cls_label_path : /home/aistudio/eval_train_weather_imblearn.txt\n",
      "[2022/01/18 00:17:24] root INFO:             image_root : /home/aistudio/\n",
      "[2022/01/18 00:17:24] root INFO:             name : ImageNetDataset\n",
      "[2022/01/18 00:17:24] root INFO:             transform_ops : \n",
      "[2022/01/18 00:17:24] root INFO:                 DecodeImage : \n",
      "[2022/01/18 00:17:24] root INFO:                     channel_first : False\n",
      "[2022/01/18 00:17:24] root INFO:                     to_rgb : True\n",
      "[2022/01/18 00:17:24] root INFO:                 ResizeImage : \n",
      "[2022/01/18 00:17:24] root INFO:                     resize_short : 256\n",
      "[2022/01/18 00:17:24] root INFO:                 CropImage : \n",
      "[2022/01/18 00:17:24] root INFO:                     size : 224\n",
      "[2022/01/18 00:17:24] root INFO:                 NormalizeImage : \n",
      "[2022/01/18 00:17:24] root INFO:                     mean : [0.5, 0.5, 0.5]\n",
      "[2022/01/18 00:17:24] root INFO:                     order : \n",
      "[2022/01/18 00:17:24] root INFO:                     scale : 1.0/255.0\n",
      "[2022/01/18 00:17:24] root INFO:                     std : [0.5, 0.5, 0.5]\n",
      "[2022/01/18 00:17:24] root INFO:         loader : \n",
      "[2022/01/18 00:17:24] root INFO:             num_workers : 4\n",
      "[2022/01/18 00:17:24] root INFO:             use_shared_memory : True\n",
      "[2022/01/18 00:17:24] root INFO:         sampler : \n",
      "[2022/01/18 00:17:24] root INFO:             batch_size : 128\n",
      "[2022/01/18 00:17:24] root INFO:             drop_last : False\n",
      "[2022/01/18 00:17:24] root INFO:             name : DistributedBatchSampler\n",
      "[2022/01/18 00:17:24] root INFO:             shuffle : False\n",
      "[2022/01/18 00:17:24] root INFO:     Train : \n",
      "[2022/01/18 00:17:24] root INFO:         dataset : \n",
      "[2022/01/18 00:17:24] root INFO:             cls_label_path : /home/aistudio/train_train_weather_imblearn.txt\n",
      "[2022/01/18 00:17:24] root INFO:             image_root : /home/aistudio\n",
      "[2022/01/18 00:17:24] root INFO:             name : ImageNetDataset\n",
      "[2022/01/18 00:17:24] root INFO:             transform_ops : \n",
      "[2022/01/18 00:17:24] root INFO:                 DecodeImage : \n",
      "[2022/01/18 00:17:24] root INFO:                     channel_first : False\n",
      "[2022/01/18 00:17:24] root INFO:                     to_rgb : True\n",
      "[2022/01/18 00:17:24] root INFO:                 RandCropImage : \n",
      "[2022/01/18 00:17:24] root INFO:                     size : 224\n",
      "[2022/01/18 00:17:24] root INFO:                 RandFlipImage : \n",
      "[2022/01/18 00:17:24] root INFO:                     flip_code : 1\n",
      "[2022/01/18 00:17:24] root INFO:                 NormalizeImage : \n",
      "[2022/01/18 00:17:24] root INFO:                     mean : [0.5, 0.5, 0.5]\n",
      "[2022/01/18 00:17:24] root INFO:                     order : \n",
      "[2022/01/18 00:17:24] root INFO:                     scale : 1.0/255.0\n",
      "[2022/01/18 00:17:24] root INFO:                     std : [0.5, 0.5, 0.5]\n",
      "[2022/01/18 00:17:24] root INFO:         loader : \n",
      "[2022/01/18 00:17:24] root INFO:             num_workers : 4\n",
      "[2022/01/18 00:17:24] root INFO:             use_shared_memory : True\n",
      "[2022/01/18 00:17:24] root INFO:         sampler : \n",
      "[2022/01/18 00:17:24] root INFO:             batch_size : 160\n",
      "[2022/01/18 00:17:24] root INFO:             drop_last : False\n",
      "[2022/01/18 00:17:24] root INFO:             name : DistributedBatchSampler\n",
      "[2022/01/18 00:17:24] root INFO:             shuffle : True\n",
      "[2022/01/18 00:17:24] root INFO: Global : \n",
      "[2022/01/18 00:17:24] root INFO:     checkpoints : None\n",
      "[2022/01/18 00:17:24] root INFO:     device : gpu\n",
      "[2022/01/18 00:17:24] root INFO:     epochs : 120\n",
      "[2022/01/18 00:17:24] root INFO:     eval_during_train : True\n",
      "[2022/01/18 00:17:24] root INFO:     eval_interval : 1\n",
      "[2022/01/18 00:17:24] root INFO:     image_shape : [3, 224, 224]\n",
      "[2022/01/18 00:17:24] root INFO:     output_dir : ./output_weather/\n",
      "[2022/01/18 00:17:24] root INFO:     pretrained_model : ./output_weather/ViT_base_patch16_224/best_model\n",
      "[2022/01/18 00:17:24] root INFO:     print_batch_step : 10\n",
      "[2022/01/18 00:17:24] root INFO:     save_inference_dir : ./inference\n",
      "[2022/01/18 00:17:24] root INFO:     save_interval : 1\n",
      "[2022/01/18 00:17:24] root INFO:     use_visualdl : False\n",
      "[2022/01/18 00:17:24] root INFO: Infer : \n",
      "[2022/01/18 00:17:24] root INFO:     PostProcess : \n",
      "[2022/01/18 00:17:24] root INFO:         class_id_map_file : ppcls/utils/imagenet1k_label_list.txt\n",
      "[2022/01/18 00:17:24] root INFO:         name : Topk\n",
      "[2022/01/18 00:17:24] root INFO:         topk : 5\n",
      "[2022/01/18 00:17:24] root INFO:     batch_size : 10\n",
      "[2022/01/18 00:17:24] root INFO:     infer_imgs : docs/images/whl/demo.jpg\n",
      "[2022/01/18 00:17:24] root INFO:     transforms : \n",
      "[2022/01/18 00:17:24] root INFO:         DecodeImage : \n",
      "[2022/01/18 00:17:24] root INFO:             channel_first : False\n",
      "[2022/01/18 00:17:24] root INFO:             to_rgb : True\n",
      "[2022/01/18 00:17:24] root INFO:         ResizeImage : \n",
      "[2022/01/18 00:17:24] root INFO:             resize_short : 256\n",
      "[2022/01/18 00:17:24] root INFO:         CropImage : \n",
      "[2022/01/18 00:17:24] root INFO:             size : 224\n",
      "[2022/01/18 00:17:24] root INFO:         NormalizeImage : \n",
      "[2022/01/18 00:17:24] root INFO:             mean : [0.5, 0.5, 0.5]\n",
      "[2022/01/18 00:17:24] root INFO:             order : \n",
      "[2022/01/18 00:17:24] root INFO:             scale : 1.0/255.0\n",
      "[2022/01/18 00:17:24] root INFO:             std : [0.5, 0.5, 0.5]\n",
      "[2022/01/18 00:17:24] root INFO:         ToCHWImage : None\n",
      "[2022/01/18 00:17:24] root INFO: Loss : \n",
      "[2022/01/18 00:17:24] root INFO:     Eval : \n",
      "[2022/01/18 00:17:24] root INFO:         CELoss : \n",
      "[2022/01/18 00:17:24] root INFO:             weight : 1.0\n",
      "[2022/01/18 00:17:24] root INFO:     Train : \n",
      "[2022/01/18 00:17:24] root INFO:         CELoss : \n",
      "[2022/01/18 00:17:24] root INFO:             weight : 1.0\n",
      "[2022/01/18 00:17:24] root INFO: Metric : \n",
      "[2022/01/18 00:17:24] root INFO:     Eval : \n",
      "[2022/01/18 00:17:24] root INFO:         TopkAcc : \n",
      "[2022/01/18 00:17:24] root INFO:             topk : [1, 2]\n",
      "[2022/01/18 00:17:24] root INFO:     Train : \n",
      "[2022/01/18 00:17:24] root INFO:         TopkAcc : \n",
      "[2022/01/18 00:17:24] root INFO:             topk : [1, 2]\n",
      "[2022/01/18 00:17:24] root INFO: Optimizer : \n",
      "[2022/01/18 00:17:24] root INFO:     lr : \n",
      "[2022/01/18 00:17:24] root INFO:         decay_epochs : [10, 22, 30]\n",
      "[2022/01/18 00:17:24] root INFO:         learning_rate : 0.01\n",
      "[2022/01/18 00:17:24] root INFO:         name : Piecewise\n",
      "[2022/01/18 00:17:24] root INFO:         values : [0.01, 0.001, 0.0001, 1e-05]\n",
      "[2022/01/18 00:17:24] root INFO:     momentum : 0.9\n",
      "[2022/01/18 00:17:24] root INFO:     name : Momentum\n",
      "[2022/01/18 00:17:24] root INFO:     regularizer : \n",
      "[2022/01/18 00:17:24] root INFO:         coeff : 0.0001\n",
      "[2022/01/18 00:17:24] root INFO:         name : L2\n",
      "[2022/01/18 00:17:24] root INFO: train with paddle 2.2.1 and device CUDAPlace(0)\n",
      "W0118 00:17:24.490124  2035 device_context.cc:447] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 10.1, Runtime API Version: 10.1\n",
      "W0118 00:17:24.495002  2035 device_context.cc:465] device: 0, cuDNN Version: 7.6.\n",
      "[2022/01/18 00:17:35] root INFO: [Eval][Epoch 0][Iter: 0/6]CELoss: 0.17755, loss: 0.17755, top1: 0.97656, top2: 1.00000, batch_cost: 5.94345s, reader_cost: 5.49357, ips: 21.53630 images/sec\n",
      "[2022/01/18 00:17:39] root INFO: [Eval][Epoch 0][Avg]CELoss: 0.54770, loss: 0.54770, top1: 0.83700, top2: 0.95154\n"
     ]
    }
   ],
   "source": [
    "# 模型评估\r\n",
    "%cd ~/PaddleClas/\r\n",
    "\r\n",
    "!python  tools/eval.py \\\r\n",
    "        -c  ./ppcls/configs/ImageNet/VisionTransformer/ViT_base_patch16_224_weather.yaml \\\r\n",
    "        -o Global.pretrained_model=./output_weather/ViT_base_patch16_224/best_model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 五、预测"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 1.时间模型导出"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/aistudio/PaddleClas\n",
      "[2022/01/18 00:34:27] root INFO: \n",
      "===========================================================\n",
      "==        PaddleClas is powered by PaddlePaddle !        ==\n",
      "===========================================================\n",
      "==                                                       ==\n",
      "==   For more info please go to the following website.   ==\n",
      "==                                                       ==\n",
      "==       https://github.com/PaddlePaddle/PaddleClas      ==\n",
      "===========================================================\n",
      "\n",
      "[2022/01/18 00:34:27] root INFO: Arch : \n",
      "[2022/01/18 00:34:27] root INFO:     class_num : 4\n",
      "[2022/01/18 00:34:27] root INFO:     name : ViT_base_patch16_224\n",
      "[2022/01/18 00:34:27] root INFO: DataLoader : \n",
      "[2022/01/18 00:34:27] root INFO:     Eval : \n",
      "[2022/01/18 00:34:27] root INFO:         dataset : \n",
      "[2022/01/18 00:34:27] root INFO:             cls_label_path : /home/aistudio/eval_train_period_imblearn.txt\n",
      "[2022/01/18 00:34:27] root INFO:             image_root : /home/aistudio/\n",
      "[2022/01/18 00:34:27] root INFO:             name : ImageNetDataset\n",
      "[2022/01/18 00:34:27] root INFO:             transform_ops : \n",
      "[2022/01/18 00:34:27] root INFO:                 DecodeImage : \n",
      "[2022/01/18 00:34:27] root INFO:                     channel_first : False\n",
      "[2022/01/18 00:34:27] root INFO:                     to_rgb : True\n",
      "[2022/01/18 00:34:27] root INFO:                 ResizeImage : \n",
      "[2022/01/18 00:34:27] root INFO:                     resize_short : 256\n",
      "[2022/01/18 00:34:27] root INFO:                 CropImage : \n",
      "[2022/01/18 00:34:27] root INFO:                     size : 224\n",
      "[2022/01/18 00:34:27] root INFO:                 NormalizeImage : \n",
      "[2022/01/18 00:34:27] root INFO:                     mean : [0.5, 0.5, 0.5]\n",
      "[2022/01/18 00:34:27] root INFO:                     order : \n",
      "[2022/01/18 00:34:27] root INFO:                     scale : 1.0/255.0\n",
      "[2022/01/18 00:34:27] root INFO:                     std : [0.5, 0.5, 0.5]\n",
      "[2022/01/18 00:34:27] root INFO:         loader : \n",
      "[2022/01/18 00:34:27] root INFO:             num_workers : 4\n",
      "[2022/01/18 00:34:27] root INFO:             use_shared_memory : True\n",
      "[2022/01/18 00:34:27] root INFO:         sampler : \n",
      "[2022/01/18 00:34:27] root INFO:             batch_size : 128\n",
      "[2022/01/18 00:34:27] root INFO:             drop_last : False\n",
      "[2022/01/18 00:34:27] root INFO:             name : DistributedBatchSampler\n",
      "[2022/01/18 00:34:27] root INFO:             shuffle : False\n",
      "[2022/01/18 00:34:27] root INFO:     Train : \n",
      "[2022/01/18 00:34:27] root INFO:         dataset : \n",
      "[2022/01/18 00:34:27] root INFO:             cls_label_path : /home/aistudio/train_train_period_imblearn.txt\n",
      "[2022/01/18 00:34:27] root INFO:             image_root : /home/aistudio\n",
      "[2022/01/18 00:34:27] root INFO:             name : ImageNetDataset\n",
      "[2022/01/18 00:34:27] root INFO:             transform_ops : \n",
      "[2022/01/18 00:34:27] root INFO:                 DecodeImage : \n",
      "[2022/01/18 00:34:27] root INFO:                     channel_first : False\n",
      "[2022/01/18 00:34:27] root INFO:                     to_rgb : True\n",
      "[2022/01/18 00:34:27] root INFO:                 RandCropImage : \n",
      "[2022/01/18 00:34:27] root INFO:                     size : 224\n",
      "[2022/01/18 00:34:27] root INFO:                 RandFlipImage : \n",
      "[2022/01/18 00:34:27] root INFO:                     flip_code : 1\n",
      "[2022/01/18 00:34:27] root INFO:                 NormalizeImage : \n",
      "[2022/01/18 00:34:27] root INFO:                     mean : [0.5, 0.5, 0.5]\n",
      "[2022/01/18 00:34:27] root INFO:                     order : \n",
      "[2022/01/18 00:34:27] root INFO:                     scale : 1.0/255.0\n",
      "[2022/01/18 00:34:27] root INFO:                     std : [0.5, 0.5, 0.5]\n",
      "[2022/01/18 00:34:27] root INFO:         loader : \n",
      "[2022/01/18 00:34:27] root INFO:             num_workers : 4\n",
      "[2022/01/18 00:34:27] root INFO:             use_shared_memory : True\n",
      "[2022/01/18 00:34:27] root INFO:         sampler : \n",
      "[2022/01/18 00:34:27] root INFO:             batch_size : 160\n",
      "[2022/01/18 00:34:27] root INFO:             drop_last : False\n",
      "[2022/01/18 00:34:27] root INFO:             name : DistributedBatchSampler\n",
      "[2022/01/18 00:34:27] root INFO:             shuffle : True\n",
      "[2022/01/18 00:34:27] root INFO: Global : \n",
      "[2022/01/18 00:34:27] root INFO:     checkpoints : None\n",
      "[2022/01/18 00:34:27] root INFO:     device : gpu\n",
      "[2022/01/18 00:34:27] root INFO:     epochs : 120\n",
      "[2022/01/18 00:34:27] root INFO:     eval_during_train : True\n",
      "[2022/01/18 00:34:27] root INFO:     eval_interval : 1\n",
      "[2022/01/18 00:34:27] root INFO:     image_shape : [3, 224, 224]\n",
      "[2022/01/18 00:34:27] root INFO:     output_dir : ./output/\n",
      "[2022/01/18 00:34:27] root INFO:     pretrained_model : ./output/ViT_base_patch16_224/best_model\n",
      "[2022/01/18 00:34:27] root INFO:     print_batch_step : 10\n",
      "[2022/01/18 00:34:27] root INFO:     save_inference_dir : ./inference\n",
      "[2022/01/18 00:34:27] root INFO:     save_interval : 1\n",
      "[2022/01/18 00:34:27] root INFO:     use_visualdl : False\n",
      "[2022/01/18 00:34:27] root INFO: Infer : \n",
      "[2022/01/18 00:34:27] root INFO:     PostProcess : \n",
      "[2022/01/18 00:34:27] root INFO:         class_id_map_file : ppcls/utils/imagenet1k_label_list.txt\n",
      "[2022/01/18 00:34:27] root INFO:         name : Topk\n",
      "[2022/01/18 00:34:27] root INFO:         topk : 5\n",
      "[2022/01/18 00:34:27] root INFO:     batch_size : 10\n",
      "[2022/01/18 00:34:27] root INFO:     infer_imgs : docs/images/whl/demo.jpg\n",
      "[2022/01/18 00:34:27] root INFO:     transforms : \n",
      "[2022/01/18 00:34:27] root INFO:         DecodeImage : \n",
      "[2022/01/18 00:34:27] root INFO:             channel_first : False\n",
      "[2022/01/18 00:34:27] root INFO:             to_rgb : True\n",
      "[2022/01/18 00:34:27] root INFO:         ResizeImage : \n",
      "[2022/01/18 00:34:27] root INFO:             resize_short : 256\n",
      "[2022/01/18 00:34:27] root INFO:         CropImage : \n",
      "[2022/01/18 00:34:27] root INFO:             size : 224\n",
      "[2022/01/18 00:34:27] root INFO:         NormalizeImage : \n",
      "[2022/01/18 00:34:27] root INFO:             mean : [0.5, 0.5, 0.5]\n",
      "[2022/01/18 00:34:27] root INFO:             order : \n",
      "[2022/01/18 00:34:27] root INFO:             scale : 1.0/255.0\n",
      "[2022/01/18 00:34:27] root INFO:             std : [0.5, 0.5, 0.5]\n",
      "[2022/01/18 00:34:27] root INFO:         ToCHWImage : None\n",
      "[2022/01/18 00:34:27] root INFO: Loss : \n",
      "[2022/01/18 00:34:27] root INFO:     Eval : \n",
      "[2022/01/18 00:34:27] root INFO:         CELoss : \n",
      "[2022/01/18 00:34:27] root INFO:             weight : 1.0\n",
      "[2022/01/18 00:34:27] root INFO:     Train : \n",
      "[2022/01/18 00:34:27] root INFO:         CELoss : \n",
      "[2022/01/18 00:34:27] root INFO:             weight : 1.0\n",
      "[2022/01/18 00:34:27] root INFO: Metric : \n",
      "[2022/01/18 00:34:27] root INFO:     Eval : \n",
      "[2022/01/18 00:34:27] root INFO:         TopkAcc : \n",
      "[2022/01/18 00:34:27] root INFO:             topk : [1, 2]\n",
      "[2022/01/18 00:34:27] root INFO:     Train : \n",
      "[2022/01/18 00:34:27] root INFO:         TopkAcc : \n",
      "[2022/01/18 00:34:27] root INFO:             topk : [1, 2]\n",
      "[2022/01/18 00:34:27] root INFO: Optimizer : \n",
      "[2022/01/18 00:34:27] root INFO:     lr : \n",
      "[2022/01/18 00:34:27] root INFO:         decay_epochs : [10, 22, 30]\n",
      "[2022/01/18 00:34:27] root INFO:         learning_rate : 0.01\n",
      "[2022/01/18 00:34:27] root INFO:         name : Piecewise\n",
      "[2022/01/18 00:34:27] root INFO:         values : [0.01, 0.001, 0.0001, 1e-05]\n",
      "[2022/01/18 00:34:27] root INFO:     momentum : 0.9\n",
      "[2022/01/18 00:34:27] root INFO:     name : Momentum\n",
      "[2022/01/18 00:34:27] root INFO:     regularizer : \n",
      "[2022/01/18 00:34:27] root INFO:         coeff : 0.0001\n",
      "[2022/01/18 00:34:27] root INFO:         name : L2\n",
      "[2022/01/18 00:34:27] root INFO: train with paddle 2.2.1 and device CUDAPlace(0)\n",
      "W0118 00:34:27.483398  3428 device_context.cc:447] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 10.1, Runtime API Version: 10.1\n",
      "W0118 00:34:27.488457  3428 device_context.cc:465] device: 0, cuDNN Version: 7.6.\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/utils.py:77: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working\n",
      "  return (isinstance(seq, collections.Sequence) and\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpg1ueymey.py:8\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpg1ueymey.py:11\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmp64fnbu8d.py:8\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmp64fnbu8d.py:11\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmp_h9tn_a7.py:8\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmp_h9tn_a7.py:11\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpjl9_2_tk.py:8\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpjl9_2_tk.py:11\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpql2k50z9.py:8\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpql2k50z9.py:11\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpknifbg79.py:8\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpknifbg79.py:11\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpiv1rr45m.py:8\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpiv1rr45m.py:11\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpa99vj9b7.py:8\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpa99vj9b7.py:11\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmp5tt116i3.py:8\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmp5tt116i3.py:11\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpo6kgrf5v.py:8\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpo6kgrf5v.py:11\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmp919bum5s.py:8\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmp919bum5s.py:11\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpbiilsbhv.py:8\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpbiilsbhv.py:11\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\n"
     ]
    }
   ],
   "source": [
    "# 模型导出\r\n",
    "%cd ~/PaddleClas/\r\n",
    "!python tools/export_model.py -c ./ppcls/configs/ImageNet/VisionTransformer/ViT_base_patch16_224.yaml -o Global.pretrained_model=./output/ViT_base_patch16_224/best_model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 2.开始预测\n",
    "编辑 **PaddleClas/deploy/python/predict_cls.py**，按提交格式输出预测结果到文件。\n",
    "\n",
    "```\n",
    "\n",
    "def main(config):\n",
    "    cls_predictor = ClsPredictor(config)\n",
    "    image_list = get_image_list(config[\"Global\"][\"infer_imgs\"])\n",
    "\n",
    "    batch_imgs = []\n",
    "    batch_names = []\n",
    "    cnt = 0\n",
    "\n",
    "    # 保存到文件\n",
    "    f=open('/home/aistudio/result.txt', 'w')\n",
    "\n",
    "    for idx, img_path in enumerate(image_list):\n",
    "        img = cv2.imread(img_path)\n",
    "        if img is None:\n",
    "            logger.warning(\n",
    "                \"Image file failed to read and has been skipped. The path: {}\".\n",
    "                format(img_path))\n",
    "        else:\n",
    "            img = img[:, :, ::-1]\n",
    "            batch_imgs.append(img)\n",
    "            img_name = os.path.basename(img_path)\n",
    "            batch_names.append(img_name)\n",
    "            cnt += 1\n",
    "\n",
    "        if cnt % config[\"Global\"][\"batch_size\"] == 0 or (idx + 1\n",
    "                                                         ) == len(image_list):\n",
    "            if len(batch_imgs) == 0:\n",
    "                continue\n",
    "            batch_results = cls_predictor.predict(batch_imgs)\n",
    "            for number, result_dict in enumerate(batch_results):\n",
    "                filename = batch_names[number]\n",
    "                clas_ids = result_dict[\"class_ids\"]\n",
    "                scores_str = \"[{}]\".format(\", \".join(\"{:.2f}\".format(\n",
    "                    r) for r in result_dict[\"scores\"]))\n",
    "                label_names = result_dict[\"label_names\"]\n",
    "                f.write(\"{} {}\\n\".format(filename, clas_ids[0]))\n",
    "                print(\"{}:\\tclass id(s): {}, score(s): {}, label_name(s): {}\".\n",
    "                      format(filename, clas_ids, scores_str, label_names))\n",
    "            batch_imgs = []\n",
    "            batch_names = []\n",
    " \n",
    "\n",
    "    if cls_predictor.benchmark:\n",
    "        cls_predictor.auto_logger.report()\n",
    "    return\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 覆盖预测文件\r\n",
    "!cp -f ~/predict_cls.py ~/deploy/python/predict_cls.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 开始预测\r\n",
    "%cd /home/aistudio/PaddleClas/deploy\r\n",
    "!python3 python/predict_cls.py  -c configs/inference_cls.yaml -o Global.infer_imgs=/home/aistudio/test_images -o Global.inference_model_dir=../inference/  -o PostProcess.Topk.class_id_map_file=None"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/aistudio\n",
      "mv: cannot stat 'result.txt': No such file or directory\r\n"
     ]
    }
   ],
   "source": [
    "%cd ~\r\n",
    "!mv result.txt result_period.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 3.天气模型导出"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 模型导出\r\n",
    "%cd ~/PaddleClas/\r\n",
    "!python tools/export_model.py -c ./ppcls/configs/ImageNet/VisionTransformer/ViT_base_patch16_224_weather.yaml -o Global.pretrained_model=./output_weather/ViT_base_patch16_224/best_model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 4.天气预测"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 开始预测\r\n",
    "%cd /home/aistudio/PaddleClas/deploy\r\n",
    "!python3 python/predict_cls.py  -c configs/inference_cls.yaml -o Global.infer_imgs=/home/aistudio/test_images -o Global.inference_model_dir=../inference_weather/  -o PostProcess.Topk.class_id_map_file=None"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/home/aistudio\n"
     ]
    }
   ],
   "source": [
    "%cd ~\r\n",
    "!mv result.txt result_weather.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 六、合并并提交\n",
    "\n",
    "### 1.预测结果合并"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "period_list = { 0:'Dawn', 1:'Dusk', 2:'Morning', 3:'Afternoon'}\r\n",
    "weather_list =  {0:'Cloudy', 1:'Rainy', 2:'Sunny'}\r\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "结果生成完毕\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\r\n",
    "import json\r\n",
    "data_period= pd.read_csv('result_period.txt', header=None, sep=' ')\r\n",
    "data_weather= pd.read_csv('result_weather.txt', header=None, sep=' ')\r\n",
    "annotations_list=[]\r\n",
    "for i in range(len(data_period)):\r\n",
    "    temp={}\r\n",
    "    temp[\"filename\"]=\"test_images\"+\"\\\\\"+data_weather.loc[i][0]\r\n",
    "    temp[\"period\"]=period_list[data_period.loc[i][1]]\r\n",
    "    temp[\"weather\"]=weather_list[data_weather.loc[i][1]]\r\n",
    "    annotations_list.append(temp)\r\n",
    "myresult={}\r\n",
    "myresult[\"annotations\"]=annotations_list\r\n",
    "\r\n",
    "with open('result.json','w') as f:\r\n",
    "    json.dump(myresult, f)\r\n",
    "    print(\"结果生成完毕\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 2.提交并获取成绩\n",
    "下载result.json并提交，即可获得成绩\n",
    "\n",
    "![](https://ai-studio-static-online.cdn.bcebos.com/f003908e265642fd9c7dc6fada990cce6ae89f5a9c09434f809d87fc020cb45e)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 3.其他注意事项\n",
    "生成版本时提示存在无效软链接无法保存 ，可以在终端 PaddleClas 下运行下列代码清理即可。\n",
    "\n",
    "```\n",
    "for a in `find . -type l`\n",
    "do\n",
    "    stat -L $a >/dev/null 2>/dev/null\n",
    "    if [ $? -gt 0 ]\n",
    "    then\n",
    "      rm $a\n",
    "    fi\n",
    "done\n",
    "```"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "py35-paddle1.2.0"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
